Repository navigation
fix(compression): dedupe large blocks by full content - #621
Conversation
|
No actionable comments were generated in the recent review. Walkthrough
Merge Risk:
|
VickyXAI
left a comment
There was a problem hiding this comment.
Verified: full-content key fixes the prefix false-positive; compression tests pass on Node 24. Thanks!
Platform fee in local cost estimates (#622), Polymarket USD rounding (#623, thanks @A5Infinity), full-content compression dedup (#621, thanks @Tyagiquamar).
Summary
Large-block deduplication in the observation compression layer now compares full message content, so two messages that merely share a leading 200 characters are no longer collapsed into a false "same content" pointer.
Root Cause & Problem
deduplicateLargeBlocksinsrc/compression/layers/observation.tsusedmsg.content.slice(0, 200)as the block identity key. Any two long messages with the same first 200 characters were treated as identical, and the later one was replaced with[See message #N - same content]. The text after the prefix was discarded, even though it can carry distinct and important information. The emitted pointer also asserted "same content" when the content was not actually the same.Technical Solution
Use the complete
msg.contentstring as the map key. This keeps the map-based exact-match behavior and removes the prefix-only false positives, with no new dependencies. Addedsrc/compression/layers/observation.test.tswith two cases: one asserts that distinct messages sharing the first 200 characters both survive compression, and one asserts that genuinely identical blocks are still collapsed. The first case fails on the original code withexpected '[See message #1 - same content]' to contain 'BETA_UNIQUE'.Test Plan
Commands run on Windows with Node 24.22.1:
npx vitest run src/compression/layers/observation.test.tson the unfixed code: FAILED as expected, 1 failed / 1 passed, withAssertionError: expected '[See message #1 - same content]' to contain 'BETA_UNIQUE'.npx prettier --check src/compression/layers/observation.ts src/compression/layers/observation.test.ts: exit code 0,All matched files use Prettier code style!(afterprettier --writeon the same two files).npx eslint src/compression/layers/observation.ts src/compression/layers/observation.test.ts: exit code 0, no diagnostics.npm run typecheck(tsc --noEmit): exit code 0, no errors.npx vitest run src/compression/layers/observation.test.tsafter the fix: PASS, 2 passed / 2 passed.Only the two changed files were touched. No new dependencies were added. The full repository test suite was not run locally.
Summary by CodeRabbit